<!DOCTYPE html>
<!-- Copyright (C) 2020  Matthew "strager" Glazar -->
<!-- See end of file for extended copyright information. -->
<html lang="en">
  <head>
    <%- await include("../common-head.ejs.html", { title: `JS linter
    benchmarks`, description: `quick-lint-js is faster than any other JavaScript
    linter.` }) %>
    <script>
      //<%
      let fs = await import("fs");
      let url = await import("url");

      let { parseBenchmarkLSPJSON, makeBenchmarkHTML } = await importFileAsync(
        "benchmark-lsp-json-to-html.mjs");
      let seriesOptions = {
        "quick-lint-js": {hue: 0.0},
        "vscode-eslint-vue": {hue: 40.0, name: "ESLint + Vue plugin"},
        "vscode-eslint-react": {hue: 50.0, name: "ESLint + React plugin"},
        "vscode-eslint-airbnb": {hue: 60.0, name: "ESLint + Airbnb rules"},
        "vscode-eslint-vanilla": {hue: 70.0, name: "ESLint"},
        "vscode-eslint-typescript": {hue: 80.0, name: "ESLint + TypeScript plugin"},
        RSLint: {hue: 120.0},
        Flow: {hue: 180.0},
        Deno: {hue: 240.0},
        "Deno-nolint": {hue: 260.0, name: "Deno (no linting)"},
        TypeScript: {hue: 290.0},
        "TypeScript-JSX": {hue: 330.0, name: "TypeScript (JSX parser)"},
      };

      let versions = JSON.parse(fs.readFileSync("full-change-wait-express-router-js.json", "utf-8")).metadata;
      //%>
    </script>
    <link href="../main.css" rel="stylesheet" />
    <link href="benchmark.css" rel="stylesheet" />
    <style>
      .linter-name {
        color: hsla(var(--hue), 70%, 40%);
      }
      @media (prefers-color-scheme: dark) {
        .linter-name {
          color: hsla(var(--hue), 85%, 75%);
        }
      }
    </style>
  </head>

  <body class="side-bar-nav">
    <header><%- await include("../common-nav.ejs.html") %></header>

    <main>
      <p>
        Which JavaScript linter is the <strong>fastest</strong> and consumes the
        <strong>least energy</strong>? We benchmarked different JavaScript
        linters to give you the answer.
      </p>
      <ul>
        <li>
          <a href="#incremental-change-wait-express-router-js"
            >LSP: incremental-change-wait express-router.js</a
          >
        </li>
        <li>
          <a href="#incremental-change-wait-react-quickly-ch10-jsx"
            >LSP: incremental-change-wait react-quickly-ch10.jsx</a
          >
        </li>
        <li>
          <a href="#full-change-wait-express-router-js"
            >LSP: full-change-wait express-router.js</a
          >
        </li>
        <li><a href="#methodology">Methodology</a></li>
        <li><a href="#comments">Comments</a></li>
      </ul>

      <article class="benchmark" id="incremental-change-wait-express-router-js">
        <h2>LSP: incremental-change-wait express-router.js</h2>
        <p>
          An <dfn>LSP server</dfn> is software which plugs into your code
          editor. It provides diagnostics (error squigglies), go-to-definition,
          and auto-complete. This benchmark measures how long an LSP server
          takes to give diagnostics after making a change in a JavaScript file
          (e.g. after typing a character). In other words, it measures
          <em>input latency</em>. Lower latency means a faster-feeling editor.
        </p>
        <h3>Results</h3>
        <%-
        makeBenchmarkHTML(parseBenchmarkLSPJSON("incremental-change-wait-express-router-js.json",
        seriesOptions)); %>
        <h3>Setup (untimed)</h3>
        <ol>
          <li>Start the LSP server.</li>
          <li>Wait for initialization to finish.</li>
          <li>
            Open one document with contents from
            <a
              href="https://github.com/expressjs/express/blob/508936853a6e311099c9985d4c11a4b1b8f6af07/lib/router/index.js"
              >express-router.js</a
            >.
          </li>
          <li>Wait for diagnostics.</li>
        </ol>

        <h3>Work (timed)</h3>
        <p>Repeat the following steps <var>N</var> times:</p>
        <ol>
          <li>
            Change a few characters in the document, sending small deltas in an
            LSP message.
          </li>
          <li>Wait for diagnostics.</li>
        </ol>
      </article>

      <article
        class="benchmark"
        id="incremental-change-wait-react-quickly-ch10-jsx"
      >
        <h2>LSP: incremental-change-wait react-quickly-ch10.jsx</h2>
        <p>
          This benchmark measures how long an LSP server takes to give
          diagnostics to an editor.
        </p>
        <p>
          This benchmark is similar to the
          <a href="#incremental-change-wait-express-router-js"
            >incremental-change-wait benchmark</a
          >, except it uses a JSX (React) source file instead of a vanilla
          JavaScript source file.
        </p>
        <h3>Results</h3>
        <%-
        makeBenchmarkHTML(parseBenchmarkLSPJSON("incremental-change-wait-react-quickly-ch10-jsx.json",
        seriesOptions)); %>
        <h3>Setup (untimed)</h3>
        <ol>
          <li>Start the LSP server.</li>
          <li>Wait for initialization to finish.</li>
          <li>
            Open one document with contents from
            <a
              href="https://github.com/azat-co/react/blob/03f39f4fe5dffcc72fc9cfc4d7a984d9f4b2aa75/ch10/nile/app.jsx"
              >react-quickly-ch10.jsx</a
            >.
          </li>
          <li>Wait for diagnostics.</li>
        </ol>

        <h3>Work (timed)</h3>
        <p>Repeat the following steps <var>N</var> times:</p>
        <ol>
          <li>
            Add or remove a few characters in the document, sending small deltas
            in an LSP message.
          </li>
          <li>Wait for diagnostics.</li>
        </ol>
      </article>

      <article class="benchmark" id="full-change-wait-express-router-js">
        <h2>LSP: full-change-wait express-router.js</h2>
        <p>
          This benchmark measures how long an LSP server takes to give
          diagnostics to an editor.
        </p>
        <p>
          This benchmark differs from the
          <a href="#incremental-change-wait-express-router-js"
            >incremental-change-wait benchmark</a
          >
          in that the editor gives the new version of the file entirely rather
          than just the incremental changes. This is for compatibility with
          linters such as RSLint which do not support incremental changes.
        </p>
        <h3>Results</h3>
        <%-
        makeBenchmarkHTML(parseBenchmarkLSPJSON("full-change-wait-express-router-js.json",
        seriesOptions)); %>
        <h3>Setup (untimed)</h3>
        <ol>
          <li>Start the LSP server.</li>
          <li>Wait for initialization to finish.</li>
          <li>
            Open one document with contents from
            <a
              href="https://github.com/expressjs/express/blob/508936853a6e311099c9985d4c11a4b1b8f6af07/lib/router/index.js"
              >express-router.js</a
            >.
          </li>
          <li>Wait for diagnostics.</li>
        </ol>

        <h3>Work (timed)</h3>
        <p>Repeat the following steps <var>N</var> times:</p>
        <ol>
          <li>
            Change a few characters in the document, sending the entire new
            document in an LSP message.
          </li>
          <li>Wait for diagnostics.</li>
        </ol>
      </article>

      <article id="methodology">
        <h2>Methodology</h2>
        <p>These benchmarks measure the following linters:</p>
        <ul>
          <li>
            <strong class="linter-name" style="--hue: 0">quick-lint-js</strong>
            version <%= versions["quick-lint-js"]["version"] %> (Debian package)
          </li>
          <li>
            <strong class="linter-name" style="--hue: 120">RSLint</strong>
            version v0.3.0 (with rustc version 1.56.1)
          </li>
          <li>
            <strong class="linter-name" style="--hue: 180">Flow</strong>
            version <%= versions["Flow"]["flow-bin"] %>
          </li>
          <li>
            <strong class="linter-name" style="--hue: 60">ESLint</strong>
            version v<%= versions["ESLint"]["eslint"] %>
            <ul>
              <li>
                @typescript-eslint/eslint-plugin version <%=
                versions["ESLint"]["@typescript-eslint/eslint-plugin"] %>
              </li>
              <li>
                @typescript-eslint/parser version <%=
                versions["ESLint"]["@typescript-eslint/parser"] %>
              </li>
              <li>
                eslint-plugin-vue version <%=
                versions["ESLint"]["eslint-plugin-vue"] %>
              </li>
              <li>
                <% let vscodeESLintURL = versions["ESLint"]["vscode-eslint"];
                let vscodeESLintCommit =
                vscodeESLintURL.match(/[0-9a-f]{40}/)[0]; %> vscode-eslint
                commit <%= vscodeESLintCommit.substr(0, 20) %>&shy;<%=
                vscodeESLintCommit.substr(20) %>
              </li>
              <li>Node version <%= versions["ESLint"]["node"] %></li>
            </ul>
          </li>
          <li>
            <strong class="linter-name" style="--hue: 240">Deno</strong>
            version <%= versions["Deno"]["deno"] %> (with V8 version <%=
            versions["Deno"]["v8"] %> and TypeScript version <%=
            versions["Deno"]["typescript"] %>)
          </li>
          <li>
            <strong class="linter-name" style="--hue: 300">TypeScript</strong>
            version <%= versions["TypeScript"]["typescript"] %> (with language
            server version <%=
            versions["TypeScript"]["typescript-language-server"] %> and Node
            version <%= versions["TypeScript"]["node"] %>)
          </li>
        </ul>

        <p>These benchmarks were measured on the following machine:</p>
        <ul>
          <li>strager's "straglum"; Purism Librem 13 Version 4 laptop</li>
          <li>CPU: Intel Core i7-7500U CPU @ 2.70 GHz (2 cores, 4 threads)</li>
          <li>
            OS: Linux Mint 20 Ulyana; Linux 5.4.0-42-generic #46-Ubuntu SMP
            x86_64 GNU/Linux
          </li>
          <li>Performance governor: performance; 400 MHz - 3.50 GHz</li>
        </ul>
      </article>

      <article id="comments">
        <h2>Comments</h2>
        <h3>Deno</h3>
        <p>
          Deno's LSP server (and thus its Visual Studio Code extension)
          <a
            href="https://github.com/denoland/deno/blob/80a9a37df1607c4713538c422309801e25024bff/cli/lsp/diagnostics.rs#L148-L151"
            >delays processing by 200 milliseconds</a
          >. This means that Deno appears to be much slower than it actually is,
          but this artificial latency does affect the editing experience.
        </p>
        <h3>TypeScript</h3>
        <p>
          TypeScript's LSP server (but not its Visual Studio Code extension)
          delays processing by
          <a
            href="https://github.com/typescript-language-server/typescript-language-server/blob/103f00c0d969bf00345ae9f5f8c2d1882c710a01/src/lsp-server.ts#L250"
            >200 milliseconds</a
          >
          +
          <a
            href="https://github.com/typescript-language-server/typescript-language-server/blob/103f00c0d969bf00345ae9f5f8c2d1882c710a01/src/diagnostic-queue.ts#L33"
            >50 milliseconds</a
          >. This means that TypeScript appears to be much slower than it
          actually is, but this artificial latency does affect the editing
          experience (in all editors except Visual Studio Code).
        </p>
      </article>
    </main>

    <footer><%- await include("../common-footer-nav.ejs.html") %></footer>

    <script type="module" src="benchmark.mjs"></script>
  </body>
</html>

<!--
quick-lint-js finds bugs in JavaScript programs.
Copyright (C) 2020  Matthew "strager" Glazar

This file is part of quick-lint-js.

quick-lint-js is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

quick-lint-js is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with quick-lint-js.  If not, see <https://www.gnu.org/licenses/>.
-->
